Learning-type movement control apparatus, method therefor, and distribution medium therefor

ABSTRACT

A learning-type movement control apparatus that learns the movement of an operation control device, predicts the movement thereof, and drives it so as to automatically move. The apparatus comprises an operation control device having a predetermined portion that is displaced according to a force exerted in an arbitrary direction, outputs the amount of the displacement at least as one-dimensional position-representing information, receives a feedback signal carrying information generated by adding displacement information to the position-representing information, and drives the predetermined portion according to a direction and a displacement that are based on the feedback signal. The apparatus also includes a learning section that receives the position-representing information and performs learning of the movement of the operation control device. The apparatus further includes a predicting section that performs prediction of a displacement of the operation control device according to the position-representing information the learning result of the learning section, and generates the feedback signal by adding the displacement information to the position-representing information.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a learning-type movement controlapparatus that has functions of learning force-input time seriespatterns and performing force feedback based on the learning, wherein aforce-input section and a control system interactively function.

Furthermore, the invention relates to a method for implementing thefunctions in the aforementioned learning-type movement controlapparatus.

Still furthermore, the present invention relates to a distributionmedium for a computer-readable program that allows an informationprocessor to execute the aforementioned method.

2. Description of the Related Art

Conventionally, an operation control device, such as a joystick or asteering wheel, is used by a user to move a predetermined portion of amovable object. The object may be a transportation means, such as afour-wheel vehicle, a two-wheel vehicle, or an aircraft; or the objectmay be a cursor on the screen of an information processor. To move theobject as the user desires, the user must operate the operation controldevice to issue operation commands to the object. To issue the operationcommands, the user must exert predetermined forces on the operationcontrol device in directions along which the user wishes to move theobject. Namely, the user must determine options, such as the directionand the distance of the movement of the object.

In the conventional device as described above, when the above isconsidered on the side of the object to be moved, the operation commandmust be issued each time the user attempts to operate the operationcontrol device, such as the steering wheel or the joystick. For the userto operate the object to meet his or her desire, and even whensubstantially the same movement is performed, the user mustintermittently or continually operate the operation control device. Thatis, since the conventional device has no learning function in theoperation control device, even to repeat a movement, the user mustperform complicated and time-consuming operations with the operationcontrol device. This is disadvantageous.

SUMMARY OF THE INVENTION

The present invention is made under the above-described circumstances.Accordingly, objects thereof are to provide:

a learning-type movement control apparatus that has the functions oflearning the movement of an operation control device, predicting themovement thereof, and driving it to automatically move;

a method implementing the functions of the aforementioned learning-typemovement control apparatus; and

a medium for distributing a computer-readable program that allows aninformation processor to execute the aforementioned method.

To achieve the aforementioned objects, according to one aspect of thepresent invention, a learning-type movement control apparatus comprisesan operation control device, a learning section, and a predictingsection. The operation control device has a predetermined portion thatis displaced according to a force exerted in an arbitrary direction. Theoperation control device outputs the amount of the displacement at leastas one-dimensional position-representing information, receives afeedback signal carrying information generated by adding displacementinformation to the position-representing information, and drives thepredetermined portion according to a direction and a displacement thatare based on the feedback signal. The learning section receives theposition-representing information, which is input in time series fromthe operation control device, and performs learning of the movement ofthe operation control device. The predicting section performs predictionof a displacement of the operation control device according to theposition-representing information that is input in time series from theoperation control device and a result of the learning by the learningsection, and generates the feedback signal by adding the displacementinformation to the position-representing information, and outputs thefeedback signal to the operation control device.

The learning section and the predicting section may be included in arecurrent neural network that comprises an input layer, a hidden layer,and an output layer. The recurrent neural network may be of a type thatperforms feedback from the output layer to the input layer. In addition,the learning section may be arranged to perform the learning of themovement of the predetermined portion of the operation control deviceaccording to an error propagation method.

According to another aspect of the invention, a learning-type movementcontrol method comprises three processing steps. A first step performslearning of a pattern of a time-series force input to an operationcontrol device by using a recurrent neural network. A second stepperforms prediction of information regarding movement of a predeterminedportion of the operation control device by using the recurrent neuralnetwork according to a result of the learning. A third step drives theoperation control device to move according to the information regardingthe movement, which has been obtained as a result of the prediction.

According to still another aspect of the present invention, adistribution medium for distributing a computer-readable program thatallows an information-processing unit to execute processing whichcomprises three steps is provided. A first step performs learning of apattern of a time-series force input to an operation control device byusing a recurrent neural network. A second step performs prediction ofinformation regarding movement of a predetermined portion of theoperation control device by using the recurrent neural network accordingto a result of the learning. A third step drives the operation controldevice to move according to the information regarding the movement,which has been obtained as a result of the prediction.

Thus, according to the present invention, for example, a user exerts adesired force on the operation control device in a desired direction.The operation control device is thereby operated so as to perform themovement desired by the user. According to the exerted force and thedirection thereof, the predetermined portion of the operation controldevice is displaced. The amount of the displacement is separated intotwo-dimensional position-representing information items (x, y). Then,the position-representing information items (x, y) are output in timeseries to the learning section and the predicting section.

The learning section receives the position-representing informationitems (x, y), and performs, for example, learning of movements of theoperation control device a predetermined number of times. It learns themovements according to, for example, an error propagation method.

According to the position-representing information items (x, y) and aresult of the learning by the learning section, the predicting sectionperforms prediction of a position the operation control device at asubsequent time. Then, the predicting section adds displacementinformation items to the position-representing information items (x, y),thereby generates a feedback signal, and outputs it to the operationcontrol device.

Having received the feedback signal from the predicting section, andaccording to directions and displacements specified by the feedbacksignal, the predetermined portion of the operation control device isdriven in either the positive direction or the negative direction ofeach of the x and y directions.

The above-described operations are iterated, and during the iteration,the movement of the operation control device is incrementally learned bythe learning section and is predicted by the predicting section. Thisallows the predetermined portion of the operation control device tooperate according to the prediction without a force being exerted by theuser.

In the above state, that is, when the predetermined portion of theoperation control device operates according to only the instructiongiven by the feedback signal, the user can exert a desired force in adesired direction on the operation control device. Accordingly, theposition-representing information items (x, y) regarding the movement atthe particular time are output in time series.

Then, according to the interaction of the learning section, thepredicting section, and the driving section, the new movement islearned. The result of the learning allows prediction information to beprovided. According to the prediction information, without a force beingexerted by the user, the predetermined portion of the operation controldevice becomes enabled to implement the new movement.

Thus, the present invention is advantageous in that it provides apractically effective system in which, without complicated operationsbeing performed, the movement of the operation control device islearned, and the movement is thereby predicted to allow the operationcontrol device to automatically move.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows the configuration of an embodiment of a learning-type forcefeedback interactive system according to the present invention;

FIG. 2 shows an example configuration of a recurrent neural networkaccording to the embodiment;

FIG. 3 is a view used to explain the operation of a learning sectionconnected to the recurrent neural network according to the embodiment;

FIG. 4 is a view used to explain the operation of a predicting sectionconnected to the recurrent neural network according to the embodiment;and

FIGS. 5A to 5C are views used to explain the operation of thelearning-type force feedback interactive system according to theembodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 is a schematic view of a learning-type force feedback interactivesystem 10. The learning-type force feedback interactive system 10 isreferred to as an embodiment of a movement control apparatus thatemploys a movement control method according to the present invention.

The learning-type force feedback interactive system 10 comprises asprimary components a joystick 11 (operation control device), a learningsection 12, a predicting section 13, and a shared memory 14.

The joystick 11 functions to output a signal S11 to the learning section12 and the predicting section 13. When a force is exerted by, forexample, a user, on a stick portion 111 in a desired direction, apredetermined portion is displaced according to the force. Thepredetermined portion is not shown in the drawing, but it can be, forexample, an end section in a main body 11 a. The amount of thedisplacement is separated into position-representing information items(x, y) in two-dimensional rectangular coordinate system, as defined andshown in FIG. 1. The position-representing information items (x, y) areoutput in time-series as a signal S11 to the learning section 12 and thepredicting section 13.

The joystick 11 includes, for example, a servomotor (not shown) in adriving section 112. The driving section 112 drives the displaceablepredetermined portion according to the direction and the displacementspecified by a signal from the predicting section 13 in either thepositive direction or the negative direction of each of the x and ydirections.

The predicting section 13 issues a feedback signal S13 which is suppliedto the driving section 112 of the joystick 11. The feedback signal S13represents information generated by adding displacement informationitems (Δx, Δy) to the position-representing information items (x, y).

As described above, the learning section 12 receives the signal S11including the position-representing information items (x, y), which havebeen supplied in time series from the joystick 11. In response to thesignal S11, the learning section 12 performs so-called online learning.It performs learning on the two-dimensional movements of the joystick 11and stores the result of the learning in the shared memory 14. Inaddition, according to the information stored therein, the learningsection 12 iterates the learning a predetermined number of times.

According to the signal S11 including current-time position-representinginformation items (x, y)(=(x_(t), y_(t))) and the learning resultsstored in the shared memory 14, the predicting section 13 predicts asubsequent-time position. Then, it adds the displacement informationitems (Δx, Δy) to the position-representing information items (x, y).Thereby, the predicting section 13 outputs the addition result to thedriving section 112 of the joystick 11 via the feedback signal S13. Thefeedback signal S13 therefore carries the information (x+Δx,y+Δy)(=(x_(t+1), y_(t+1))).

The learning section 12, the predicting section 13, and the sharedmemory 14 are provided, for example, on the side of a computer main unit(not shown). In reality, they function as primary components that areincluded (connected to) in a recurrent neural network RNN.

FIG. 2 shows an example of the recurrent neural network with referenceto symbol RNN.

As shown in the figure, the recurrent neural network RNN has ahierarchical structure. It comprises an input layer 21, a hidden layer22, and an output layer 23, each of which is described below.

The input layer 21 includes a predetermined number of neurons, forexample, four neurons 211 to 214. Among the neurons 211 to 214, the twoneurons 211 and 212 receives the position-representing information itemsx and y as inputs, respectively. As already described, theposition-representing information items x and y are supplied in timeseries by the output signal S11 of the joystick 11. The two otherneurons 213 and 214 are used as feedback neurons from so called contextin the output layer 23.

The hidden layer 22 includes a predetermined number of neurons, forexample, seven neurons 221 to 227. The neurons 221 to 227 randomlyreceive outputs of the individual neurons 211 to 214 in the input layer21.

The output layer 23 includes a predetermined number of neurons, forexample, four neurons 231 to 234. Among the neurons 231 to 234, the twoneurons 231 and 232 individually incorporate the outputs of the neuronsin the hidden layer 22, and generate the displacement information itemsΔx and Δy, respectively. As already described, the displacementinformation items Δx and Δy relate to the position-representinginformation items x and y of feedback signal S13, respectively, whichare output to the joystick 11. The two other neurons 233 and 234 areused as the feedback neurons that will be sent to the input layer 21.

As a result of, for example, the learning, each of the neurons memorizesa predetermined weight coefficient. It multiplies an input by the weightcoefficient, and outputs the multiplication result to another neuron orneurons.

Hereinbelow, practical examples of operations of the learning section 12and the predicting section 13, which will be performed in the recurrentneural network RNN.

First of all, as shown in FIG. 3, the learning section 12 in therecurrent neural network RNN execute a rehearsal sequence. Then, itperforms learning.

A. Rehearsal Sequence

(1) Random initialization with values between 0 and 1 is performed forinput units 211 and 212 (input values thereof) in the input layer 21 andcontext units of the recurrent neural network RNN.

(2) The recurrent neural network RNN is set to a closed-loop mode inwhich outputs are applied to inputs as self-feedback, and an N-stepsequence is generated from initial values produced after theinitialization.

(3) The initialization mentioned in (1) and the generating processingmentioned in (2) are iterated L times, and L rows of rehearsal sequencesare thereby obtained.

B. Learning

(1) The aforementioned L rows of rehearsal sequences and one row ofexperience sequence are added together, and (L+1) rows of learningsequences are thereby prepared.

(2) The aforementioned rows of learning sequences are learned M times inthe recurrent neural network RNN according to, for example, an errorpropagation method (Reference: D. E. Rumelhart et al., “ParallelDistributed Processing”, MIT Press, 1986). Thereby, a weights matrixstored in the shared memory 14 is updated.

When the error propagation method is used, two learning methods can beperformed. In one method, when the error between an output obtainedaccording to a pattern provided to the input layer 21 and a patternrequired for the output layer 23 becomes equal to or less than apredetermined value, the learning is terminated. In the other method,the learning is terminated after it is iterated a predetermined numberof times.

Hereinbelow, the performance of the predicting section 13 will bedescribed below with reference to FIG. 4.

In the predicting section 13, input nodes receive theposition-representing information items (x, y)(=(xt, yt)) regarding thepredetermined portion of the joystick 11 at the current time. Outputvalues (Δx, Δy) of the recurrent neural network RNN are received asdisplacement information items. Then, the feedback signal S13 includingthe information (x+Δx, y+Δy)(=(x_(t+1), y_(t+1))), which has beenobtained by adding the displacement information items (Δx, Δy) to theposition-representing information items (x, y), is output to thejoystick 11, which is provided as the operation control device.

Hereinbelow, a description will be given of the operation performed bythe above-described configuration.

First of all, in the recurrent neural network RNN, random initializationis performed for the input neurons 211 and 212 (input values thereof) inthe input layer 21 and context units (neurons) 213, 214, 233, and 234 ofthe recurrent neural network RNN. The recurrent neural network RNN isset to a closed-loop mode, and an N-step sequence is generated frompost-initialization initial values. Then, the initialization processingand the generation processing of the N-step sequence are iterated Ltimes, and L rows of rehearsal sequences can be thereby obtained.

In this state, for example, as shown in FIGS. 5A and 5B, a user (notshown) exerts a desired force in a desired direction on the stickportion 111 of the joystick 11. Thereby, the stick portion 111 is movedin such a manner as to form a figure “8”. According to the exerted forceand the direction thereof, the predetermined portion of the stickportion 111 is displaced. Subsequently, the displacement amount isseparated into the two-dimensional position-representing informationitems (x, y), and the position-representing information items (x, y) areoutput in time series as the signal S11 to the learning section 12 andthe predicting section 13.

In the learning section 12, first of all, the L rows of rehearsalsequences are added to one row of experience sequence. Namely, theposition-representing information items (x, y) regarding the joystick 11are added thereto. Thereby, (L+1) rows of learning sequences areprepared. Subsequently, in the recurrent neural network RNN, theaforementioned rows of learning sequences are learned M times accordingto, for example, the error propagation method. Thereby, the weightsmatrix stored in the shared memory 14 is updated.

In the predicting section 13, the position-representing informationitems (x, y)(=(x_(t), y_(t))) regarding the predetermined portion of thejoystick 11 at the current time are supplied. In addition, the outputvalues (Δx, Δy) of the recurrent neural network RNN are received as thedisplacement information items. Subsequently, the feedback signal S13 isgenerated and is output to the driving section 112 of the joystick 11.The feedback signal S13 includes the information items (x+Δx,y+Δy)(=(x_(t+1), y_(t+1))) obtained by adding the displacementinformation items (Δx, Δy) to the position-representing informationitems (x, y).

In the driving section 112 of the joystick 11, the feedback signal S13is received from the predicting section 13. Then, according to thedirection and the displacement that are specified by the feedback signalS13, the predetermined portion of the stick portion 111 is driven ineither the positive direction or the negative direction of each of the xand y directions.

The above-described operations are iterated, and during the iteration,as shown in FIG. 5C, the movement that is similar to forming the figure“8” is incrementally learned by the learning section 12 and is predictedby the predicting section 13. This allows the stick portion 111 of thejoystick 11 to operate according to the prediction without a force beingexerted by the user.

In the above state, that is, when the stick portion 111 of the joystick11 operates according to only the instruction given by the feedbacksignal S13, the user can exert a desired force in a desired direction onthe stick portion 111. Thereby, the stick portion 111 moves differentlyfrom the movement that is similar to forming the figure number “8”. Inthis case, the position-representing information items (x, y) regardingthe above movement are output in time series as the signal S11.

After the above, according to the interaction of the learning section12, the predicting section 13, and the driving section 112, a newmovement is learned. The result of the learning allows predictioninformation to be provided. According to the prediction information,without a force being exerted by the user, the stick portion 111 of thejoystick 11 is able to perform a new movement that is different from themovement that is similar to forming the figure “8”.

Thus, the described embodiment has the advantages that, withoutcomplicated and time-consuming operations being performed, the movementof the operation control device is learned, and the movement is therebypredicted to allow the operation control device to automaticallyoperate. The embodiment can exhibit these advantages because it has thejoystick 11, the learning section 12, and the predicting section 13.

The joystick 11 has the predetermined portion (not shown). When a forceis exerted by, for example, a user, the predetermined portion isdisplaced according to the force exerted in a direction desired by theuser. The joystick 11 separates the amount of the displacement into thetwo-dimensional position-representing information items (x, y) andoutputs them in time series as the signal S11. According to thedirection and the displacement that are based on the feedback signalS13, the joystick 11 drives the displaceable predetermined portion ineither the positive direction or the negative direction of each of the xand y directions.

The learning section 12 is connected to the recurrent neural network RNNand receives the signal S11 including the position-representinginformation items (x, y) supplied in time series from the joystick 11.In response to the signal, the learning section 12 performs the onlinelearning of the two-dimensional movement of the joystick 11 and storesthe result of the learning in the shared memory 14. In addition,according to the information stored therein, the learning section 12performs the learning a predetermined number of times.

The predicting section 13 is also connected to the recurrent neuralnetwork RNN. The predicting section 13 predicts a subsequent-timeposition according to the signal S11 including current-timeposition-representing information items (x, y)(=(x_(t), y_(t))) and thelearning results stored in the shared memory 14. Then, the predictingsection 13 adds the displacement information items (Δx, Δy) to theposition-representing information items (x, y). Thereby, the predictingsection 13 outputs the added result as a feedback signal S13 includingthe information (x+Δx, y+Δy)(=(x_(t+1), y_(t+1))) to the joystick 11.

As above, in the present embodiment, although description has been madewith reference to the example configuration where the joystick 11receives two-dimensional information items x and y, the presentinvention is not limited thereto. The invention may be configured topermit the joystick 11 to receive, for example, three-dimensionalinformation items x, y, and z. In this case, for the additionalinformation item z, the number of input neurons in the input layer 21 ofthe recurrent neural network RNN are increased by one, and similarly, aneuron for a positional-variation Δz is added to the neurons in theoutput layer 23 of the recurrent neural network RNN.

Also, although the embodiment has been described with reference to thejoystick 11 as an example control object, the invention is not limitedthereto. The invention can control a steering wheel and various otherdevices. In addition, the invention may also be applied to, for example,entertainment apparatuses and other apparatuses.

Furthermore, the above-described processing is programmed for executionwith a computer. The computer program can be distributed to users byvarious methods of distribution. For example, the program may bedistributed via various computer-readable recording media including acompact disk read only memory (CD-ROM) and a solid-state memory, and viacommunication methods including communication networks and satellites.

What is claimed is:
 1. A learning-type movement control apparatuscomprising: an operation control device that has a predetermined portionwhich is displaced according to a force exerted in an arbitrarydirection, that outputs the amount of the displacement at least asone-dimensional position-representing information, that receives afeedback signal carrying information generated by adding displacementinformation to the position-representing information, and that drivessaid predetermined portion according to a direction and a displacementthat are based on the feedback signal; a learning section that receivesthe position-representing information which is input in time series fromsaid operation control device and that performs learning of the movementof said operation control device; and a predicting section that performsprediction of the displacement of said operation control deviceaccording to the position-representing information that is input in timeseries from said operation control device and a result of the learningby said learning section, that generates the feedback signal by addingthe displacement information to the position-representing information,and that outputs the feedback signal to said operation control device.2. A learning-type movement control apparatus as claimed in claim 1,further comprising a recurrent neural network that comprises: saidlearning section; said predicting section; an input layer; a hiddenlayer; and an output layer.
 3. A learning-type movement controlapparatus as claimed in claim 2, wherein said recurrent neural networkperforms feedback from said output layer to said input layer.
 4. Alearning-type movement control apparatus as claimed in claim 1, whereinsaid learning section performs the learning of the movement of saidpredetermined portion of said operation control device according to anerror propagation method.
 5. A learning-type movement control apparatusas claimed in claim 2, wherein said learning section performs thelearning of the movement of said predetermined portion of said operationcontrol device according to an error propagation method.
 6. Alearning-type movement control apparatus as claimed in claim 3, whereinsaid learning section performs the learning of the movement of saidpredetermined portion of said operation control device according to anerror propagation method.
 7. A learning-type movement control methodcomprising the steps of: performing learning of a time-series pattern ofmovement information output by an operation control device by using arecurrent neural network, where said movement information output by saidoperation control device is based on movement of a predetermined portionof said operation control device according to a pattern of force appliedto said operation control device; performing prediction of informationregarding movement of said predetermined portion of said operationcontrol device by using said recurrent neural network according to aresult of the learning; providing a feedback signal to said operationcontrol device, where said feedback signal is the sum of the movementinformation output by the operation control device and the predictedmovement information; and driving said operation control device to moveaccording to said feedback signal.
 8. A learning-type movement controlmethod as claimed in claim 7, wherein said recurrent neural networkperforms feedback from an output layer to an input layer.
 9. Alearning-type movement control method as claimed in claim 7, whereinsaid learning section performs the learning of the movement of saidpredetermined portion of said operation control device according to anerror propagation method.
 10. A learning-type movement control method asclaimed in claim 8, wherein said learning section performs the learningof the movement of said predetermined portion of said operation controldevice according to an error propagation method.
 11. A distributionmedium for distributing a computer-readable program that allows aninformation-processing unit to execute processing comprising the stepsof: performing learning of a time-series pattern of movement informationoutput by an operation control device by using a recurrent neuralnetwork, where said movement information output by said operationcontrol device is based on movement of a predetermined portion of saidoperation control device according to a pattern of force applied to saidoperation control device; performing prediction of information regardingmovement of said predetermined portion of said operation control deviceby using said recurrent neural network according to a result of thelearning; providing a feedback signal to said operation control device,where said feedback signal is the sum of the movement information outputby the operation control device and the predicted movement information;and driving said operation control device to move according to saidfeedback signal.
 12. A learning-type movement control apparatus asclaimed in claim 1, where the operation control device is forcontrolling the movement of a position indicating item on a display in acomputer system.
 13. A learning-type movement control apparatus asclaimed in claim 3, wherein said input layer includes one or more inputcontext neurons and said output layer includes one or more outputcontext neurons, and wherein each output context neuron provides data toa corresponding input context neuron.
 14. A learning-type movementcontrol method as claimed in claim 7, where the movement of saidoperation control device controls the movement of a position indicatingitem on a display in a computer system.
 15. A learning-type movementcontrol method as claimed in claim 8, wherein said input layer includesone or more input context neurons and said output layer includes one ormore output context neurons, and wherein each output context neuronprovides data to a corresponding input context neuron.
 16. Adistribution medium as claimed in claim 11, where the movement of saidoperation control device controls the movement of a position indicatingitem on a display in a computer system.
 17. A distribution medium asclaimed in claim 11, wherein: said recurrent neural network performsfeedback from an output layer to an input layer, said input layerincludes one or more input context neurons, said output layer includesone or more output context neurons, and each output context neuronprovides data to a corresponding input context neuron.